Offsetting image light aberration due to waveguide movement in display assemblies using information from piezoelectric movement sensors

ABSTRACT

A display assembly monitors movements in a waveguide assembly and corrects for aberrations in image light caused by the monitored movements. For example, an artificial reality headset may include a display assembly that monitors for changes in shape or displacement of waveguide assemblies that generate three dimensional images for display with a real world environment. The display assembly includes movement sensors (e.g., piezoelectric movement sensors) coupled to the waveguide assembly. The movement sensors monitor the movement of the waveguide assembly and provide the monitored movement to a display controller that generates instructions for correcting aberrations in the image light.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.63/214,982, filed Jun. 25, 2021, which is incorporated by reference inits entirety.

FIELD OF THE INVENTION

This disclosure relates generally to display assemblies, and morespecifically to offsetting image light aberration due to waveguidemovement in display assemblies.

BACKGROUND

Display assemblies include one or more waveguides that provide imagecontent. However, performance of display assemblies can be greatlydegraded by even slight movements of the one or more waveguides.Examples of movements can include deformation or misalignment caused bystresses from mounting a display assembly to a larger device (e.g., anartificial reality headset), thermal stresses, electric shock, dropimpact (e.g., dropping the headset), and more. Movement can result inimage distortion, binocularity disparity, low image contrast, and otherimage display effects that reduce the accuracy with which images areprojected through the display assemblies.

SUMMARY

A display assembly is described herein that monitors movements in awaveguide assembly and corrects for aberrations in image light caused bythe monitored movements. For example, an artificial reality headset mayinclude a display assembly that monitors for changes in shape ordisplacement of waveguide assemblies that generate three dimensionalimages for display with a real world environment. The display assemblyincludes movement sensors (e.g., piezoelectric movement sensors) coupledto the waveguide assembly. The movement sensors may be sub-micronprecision sensors that are substantially transparent to both exhibithigh sensitivity to small scale movements that affect projected imagelight and provide sufficient transmission efficiency to view the realworld environment through the waveguide assembly. The movement sensorsmonitor the movement of the waveguide assembly and feed back themovement to a display controller that generates instructions forcorrecting aberrations in the image light. The instructions may includesoftware modifications such as modifying the image light beforeprojection to compensate for the deformed waveguide assembly, mechanicalmodifications such as instructing actuators of the display assembly tocorrect the shape of a deformed waveguide assembly, or a combinationthereof. Thus, the display assembly described herein improves thequality of image light generated by conventional display assemblies bymonitoring for movements in a waveguide assembly and correctingaberrations caused by the movements.

In one embodiment, a display assembly includes a waveguide assembly, oneor more piezoelectric movement sensors coupled to the waveguideassembly, and a display controller. The waveguide assembly is configuredto project image light towards an eyebox. Movement of the waveguideassembly may contribute at least in part to an amount of aberration inthe projected image light. The one or more piezoelectric movementsensors are configured to monitor the movement of the waveguideassembly. The display controller is configured to correct for the amountof aberration in the image light based in part on the monitoredmovement.

In another embodiment, a method includes projecting image light from awaveguide assembly towards an eyebox. Movement of the waveguide assemblymay contribute at least in part to an amount of aberration in theprojected image light. The method further includes monitoring, via oneor more piezoelectric movement sensors coupled to the waveguideassembly, the movement of the waveguide assembly. The amount ofaberration in the projected image light is corrected based in part onthe monitored movement.

In yet another embodiment, a non-transitory computer-readable storagemedium includes stored instructions that, when executed by a processorof a device, cause the device to project image light from a waveguideassembly towards an eyebox, monitor the movement of the waveguideassembly via one or more piezoelectric movement sensors coupled to thewaveguide assembly, and correct for an amount of aberration in the imagelight based in part on the monitored movement. The amount of aberrationin the image light can be based in part on a movement of the waveguideassembly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a perspective view of a headset implemented as an eyeweardevice, in accordance with one or more embodiments.

FIG. 1B is a perspective view of a headset of FIG. 1 showing sensors fordetecting movement of a display assembly of the headset, in accordancewith one or more embodiments.

FIG. 2 is a block diagram of a display assembly, in accordance with oneor more embodiments.

FIG. 3 is a top view and cross sectional view of a display assembly, inaccordance with one or more embodiments.

FIG. 4 depicts movement sensor configurations, in accordance withvarious embodiments.

FIG. 5 is a block diagram of a feedback process for correctingaberrations monitored by movement sensors, in accordance with one ormore embodiments.

FIG. 6 is a flowchart illustrating a process for correcting an amount ofaberration of a waveguide assembly, in accordance with one or moreembodiments.

FIG. 7 is a system that includes a headset, in accordance with one ormore embodiments.

The figures depict various embodiments for purposes of illustrationonly. One skilled in the art will readily recognize from the followingdiscussion that alternative embodiments of the structures and methodsillustrated herein may be employed without departing from the principlesdescribed herein.

DETAILED DESCRIPTION

Embodiments pertaining to a display assembly that offsets image lightaberration due to waveguide movement are described herein. The displayassembly may be part of, e.g., a headset. The display assembly monitorsand dynamically corrects for movement of the one or more waveguides. Thedisplay assembly includes one or more waveguide assemblies and movementsensors that are coupled to one or more surfaces of the one or morewaveguide assemblies. The movement sensors can be piezoelectric,piezoresistive, or a combination thereof. The movement sensors aresubstantially transparent and measure movement of the waveguideassembly. A display controller uses the measured movement to correct foraberration in the image light caused at least in part by the movement ofthe waveguide assembly. The correction may be done via software and/ormechanical movement (e.g., via actuators) of components of the displayassembly (e.g., a projector assembly or a waveguide assembly). Themovements may be detected in substantially real time (e.g., within oneto two seconds) in response to an event that is likely to cause amovement. Thus, the monitored movement can be fed to a displaycontroller to correct for a distortion or disparity in one or morewaveguide assemblies.

Display assemblies may be subject to various stressors that deform,misalign, or otherwise cause movement in one or more waveguideassemblies in the display assembly. The movement in a waveguide causesdistortion in the image displayed to the user due to aberrations inimage light that is projected through the moved waveguide assembly. Thedisplay assembly described herein improves the quality of images beingdisplayed by correcting for those aberrations. The display assembly alsohas an increased durability over a conventional display assembly thatmight otherwise be disposed of at the first sign of damage to itsdisplay capabilities.

Embodiments of the invention may include or be implemented inconjunction with an artificial reality system. Artificial reality is aform of reality that has been adjusted in some manner beforepresentation to a user, which may include, e.g., a virtual reality (VR),an augmented reality (AR), a mixed reality (MR), a hybrid reality, orsome combination and/or derivatives thereof. Artificial reality contentmay include completely generated content or generated content combinedwith captured (e.g., real-world) content. The artificial reality contentmay include video, audio, haptic feedback, or some combination thereof,any of which may be presented in a single channel or in multiplechannels (such as stereo video that produces a three-dimensional effectto the viewer). Additionally, in some embodiments, artificial realitymay also be associated with applications, products, accessories,services, or some combination thereof, that are used to create contentin an artificial reality and/or are otherwise used in an artificialreality. The artificial reality system that provides the artificialreality content may be implemented on various platforms, including awearable device (e.g., headset) connected to a host computer system, astandalone wearable device (e.g., headset), a mobile device or computingsystem, or any other hardware platform capable of providing artificialreality content to one or more viewers.

FIG. 1A is a perspective view of a headset 100 implemented as an eyeweardevice, in accordance with one or more embodiments. In some embodiments,the eyewear device is a near eye display (NED). In general, the headset100 may be worn on the face of a user such that content (e.g., mediacontent) is presented using a display assembly and/or an audio system.However, the headset 100 may also be used such that media content ispresented to a user in a different manner. Examples of media contentpresented by the headset 100 include one or more images, video, audio,or some combination thereof. The headset 100 includes a frame, and mayinclude, among other components, a display assembly 105, a depth cameraassembly (DCA), an audio system, and a position sensor 190. While FIG.1A illustrates the components of the headset 100 in example locations onthe headset 100, the components may be located elsewhere on the headset100, on a peripheral device paired with the headset 100, or somecombination thereof. Similarly, there may be more or fewer components onthe headset 100 than what is shown in FIG. 1A.

The frame 110 holds the other components of the headset 100. The frame110 includes a front part that holds the one or more display elements120 and end pieces (e.g., temples) to attach to a head of the user. Thefront part of the frame 110 bridges the top of a nose of the user. Thelength of the end pieces may be adjustable (e.g., adjustable templelength) to fit different users. The end pieces may also include aportion that curls behind the ear of the user (e.g., temple tip, earpiece).

The display assembly 120 provides light to a user wearing the headset100. The light may be light from environmental sources (e.g., lamps,sunlight, etc.) or light generated by the headset 100 (e.g., image lightgenerated by a projector assembly of the headset 100). In someembodiments, the display assembly 120 generates image light that isprovided to an eyebox of the headset 100. The eyebox is a location inspace that an eye of user occupies while wearing the headset 100. Thedisplay assembly 120 may use a projector assembly and a waveguideassembly to provide light to the user. A projector assembly includes alight source (e.g., a two-dimensional source, one or more line sources,one or more point sources, etc.). A waveguide assembly includes one ormore waveguides. Light from the light source is in-coupled into the oneor more waveguides, which outputs the light in a manner such that thereis pupil replication in an eyebox of the headset 100. In-coupling and/oroutcoupling of light from the one or more waveguides may be done usingone or more diffraction gratings. In some embodiments, the displayassembly 120 includes a scanning element (e.g., waveguide, mirror, etc.)that scans light from the light source as it is in-coupled into the oneor more waveguides. Note that in some embodiments, a portion of thedisplay assembly 120 is opaque and does not transmit light from a localarea around the headset 100. The local area is the area surrounding theheadset 100. For example, the local area may be a room that a userwearing the headset 100 is inside, or the user wearing the headset 100may be outside and the local area is an outside area. In this context,the headset 100 generates VR content. Alternatively, in someembodiments, the display assembly 120 is at least partially transparent,such that light from the local area may be combined with light from theone or more display elements to produce AR and/or MR content.

In some embodiments, the display assembly 120 does not generate imagelight, and instead is a lens that transmits light from the local area tothe eyebox. For example, the display assembly 120 may be a lens withoutcorrection (non-prescription) or a prescription lens (e.g., singlevision, bifocal and trifocal, or progressive) to help correct fordefects in a user's eyesight. In some embodiments, the display assembly120 may be polarized and/or tinted to protect the user's eyes from thesun.

In some embodiments, the display assembly 120 monitors movements in awaveguide assembly and corrects for aberrations in image light caused bythe monitored movements. For example, the display assembly 120 canmonitor for changes in shape or displacement of waveguide assembliesthat generate three dimensional images for display with a real worldenvironment. The display assembly 120 includes movement sensors (e.g.,piezoelectric movement sensors) coupled to the waveguide assembly. Themovement sensors monitor the movement of the waveguide assembly andprovide the monitored movement to a display controller that generatesinstructions for correcting aberrations in the image light.

The DCA determines depth information for a portion of a local areasurrounding the headset 100. The DCA includes one or more imagingdevices 130 and a DCA controller (not shown in FIG. 1A), and may alsoinclude an illuminator 140. In some embodiments, the illuminator 140illuminates a portion of the local area with light. The light may be,e.g., structured light (e.g., dot pattern, bars, etc.) in the infrared(IR), IR flash for time-of-flight, etc. In some embodiments, the one ormore imaging devices 130 capture images of the portion of the local areathat include the light from the illuminator 140. As illustrated, FIG. 1Ashows a single illuminator 140 and two imaging devices 130. In alternateembodiments, there is no illuminator 140 and at least two imagingdevices 130.

The DCA controller computes depth information for the portion of thelocal area using the captured images and one or more depth determinationtechniques. The depth determination technique may be, e.g., directtime-of-flight (ToF) depth sensing, indirect ToF depth sensing,structured light, passive stereo analysis, active stereo analysis (usestexture added to the scene by light from the illuminator 140), someother technique to determine depth of a scene, or some combinationthereof.

The DCA may include an eye tracking unit that determines eye trackinginformation. The eye tracking information may comprise information abouta position and an orientation of one or both eyes (within theirrespective eye-boxes). The eye tracking unit may include one or morecameras. The eye tracking unit estimates an angular orientation of oneor both eyes based on images captures of one or both eyes by the one ormore cameras. In some embodiments, the eye tracking unit may alsoinclude one or more illuminators that illuminate one or both eyes withan illumination pattern (e.g., structured light, glints, etc.). The eyetracking unit may use the illumination pattern in the captured images todetermine the eye tracking information. The headset 100 may prompt theuser to opt in to allow operation of the eye tracking unit. For example,by opting in the headset 100 may detect, store, images of the user's anyor eye tracking information of the user.

The audio system provides audio content. The audio system includes atransducer array, a sensor array, and an audio controller 150. However,in other embodiments, the audio system may include different and/oradditional components. Similarly, in some cases, functionality describedwith reference to the components of the audio system can be distributedamong the components in a different manner than is described here. Forexample, some or all of the functions of the controller may be performedby a remote server.

The transducer array presents sound to user. The transducer arrayincludes a plurality of transducers. A transducer may be a speaker 160or a tissue transducer 170 (e.g., a bone conduction transducer or acartilage conduction transducer). Although the speakers 160 are shownexterior to the frame 110, the speakers 160 may be enclosed in the frame110. In some embodiments, instead of individual speakers for each ear,the headset 100 includes a speaker array comprising multiple speakersintegrated into the frame 110 to improve directionality of presentedaudio content. The tissue transducer 170 couples to the head of the userand directly vibrates tissue (e.g., bone or cartilage) of the user togenerate sound. The number and/or locations of transducers may bedifferent from what is shown in FIG. 1A.

The sensor array detects sounds within the local area of the headset100. The sensor array includes a plurality of acoustic sensors 180. Anacoustic sensor 180 captures sounds emitted from one or more soundsources in the local area (e.g., a room). Each acoustic sensor isconfigured to detect sound and convert the detected sound into anelectronic format (analog or digital). The acoustic sensors 180 may beacoustic wave sensors, microphones, sound transducers, or similarsensors that are suitable for detecting sounds.

In some embodiments, one or more acoustic sensors 180 may be placed inan ear canal of each ear (e.g., acting as binaural microphones). In someembodiments, the acoustic sensors 180 may be placed on an exteriorsurface of the headset 100, placed on an interior surface of the headset100, separate from the headset 100 (e.g., part of some other device), orsome combination thereof. The number and/or locations of acousticsensors 180 may be different from what is shown in FIG. 1A. For example,the number of acoustic detection locations may be increased to increasethe amount of audio information collected and the sensitivity and/oraccuracy of the information. The acoustic detection locations may beoriented such that the microphone is able to detect sounds in a widerange of directions surrounding the user wearing the headset 100.

The audio controller 150 processes information from the sensor arraythat describes sounds detected by the sensor array. The audio controller150 may comprise a processor and a computer-readable storage medium. Theaudio controller 150 may be configured to generate direction of arrival(DOA) estimates, generate acoustic transfer functions (e.g., arraytransfer functions and/or head-related transfer functions), track thelocation of sound sources, form beams in the direction of sound sources,classify sound sources, generate sound filters for the speakers 160, orsome combination thereof.

The position sensor 190 generates one or more measurement signals inresponse to motion of the headset 100. The position sensor 190 may belocated on a portion of the frame 110 of the headset 100. The positionsensor 190 may include an inertial measurement unit (IMU). Examples ofposition sensor 190 include: one or more accelerometers, one or moregyroscopes, one or more magnetometers, another suitable type of sensorthat detects motion, a type of sensor used for error correction of theIMU, or some combination thereof. The position sensor 190 may be locatedexternal to the IMU, internal to the IMU, or some combination thereof.

In some embodiments, the headset 100 may provide for simultaneouslocalization and mapping (SLAM) for a position of the headset 100 andupdating of a model of the local area. For example, the headset 100 mayinclude a passive camera assembly (PCA) that generates color image data.The PCA may include one or more RGB cameras that capture images of someor all of the local area. In some embodiments, some or all of theimaging devices 130 of the DCA may also function as the PCA. The imagescaptured by the PCA and the depth information determined by the DCA maybe used to determine parameters of the local area, generate a model ofthe local area, update a model of the local area, or some combinationthereof. Furthermore, the position sensor 190 tracks the position (e.g.,location and pose) of the headset 100 within the room. Additionaldetails regarding the components of the headset 100 are discussed belowin connection with FIG. 7 .

FIG. 1B is a perspective view of a headset of FIG. 1 showing sensors fordetecting movement of a display assembly of the headset, in accordancewith one or more embodiments. The display assembly 120 may offset imagelight aberration caused by movement in a waveguide assembly 140 of thedisplay assembly 120. In particular, the display assembly 120 canmonitor and dynamically correct for movement of the waveguide assembly140. The display assembly 120 includes the waveguide assembly 140, atleast one movement sensor, and a display controller 135. The at leastone movement sensor is coupled to one or more surfaces of the waveguideassembly 140. The piezoelectric movement sensors include one or moreelectrodes 115 and a piezoelectric thin film 125. The display controller135 can use the measured movement to correct for aberration in the imagelight caused at least in part by the movement of the waveguide assembly140. The correction may be done via software, mechanical movement (e.g.,via actuators), or a combination thereof. A display assembly is furtherdescribed in the description of FIG. 2 .

FIG. 2 is a block diagram of a display assembly 200, in accordance withone or more embodiments. The display assembly 120 of FIG. 1A and FIG. 1Bmay include all or some of the components of the display assembly 200.The display assembly 200 monitors and corrects for aberrations in awaveguide assembly 220 of the display assembly 200. In the embodiment ofFIG. 2 , the display assembly 200 includes a projector assembly 210, awaveguide assembly 220, piezoelectric movement sensors 230, and andisplay controller 250. Some embodiments of the display assembly 200have additional, fewer, or different components than those describedhere. In a first example, the display assembly 200 may additionallyinclude actuators for implementing mechanical correction of a deformedshape of the waveguide assembly 220 according to instructions generatedby the display controller 250. In a second example, the display assembly200 may exclude the temperature sensor 240. Similarly, in some cases,functions can be distributed among the components in a different mannerthan is described here.

The display assembly 200 generates image light, measures movement in awaveguide through which the generated image light travels, and correctsfor the measured movement. The display assembly 200 includes a projectorassembly 210 and the waveguide assembly 220 for generating andtransporting light for reception by a user's eye(s), respectively. Thedisplay assembly 200 includes one or more piezoelectric movement sensors230 to measure movement in a waveguide. A display controller 250 of thedisplay assembly 200 corrects for an amount of aberration in the imagelight caused by the movement in the waveguide. The movement may becorrected via actuators and/or in software. A display assembly isfurther described in the description of FIG. 3 .

The projector assembly 210 can project image light into the waveguideassembly 220. The projector assembly 210 generates the light that isincoupled into one or more waveguides of the waveguide assembly 220. Thewaveguides output the light, which combine to form an image in theeyebox. The projector assembly 210 may include light sources indifferent color channels (e.g., red, green, and blue) that generateimage light. The different color channels may correspond to respectivewaveguides of the waveguide assembly 220. The projector assembly 210 cangenerate image light in accordance with instructions from the displaycontroller 250. The projector assembly 210 generates at least a coherentor partially coherent image light. The projector assembly 210 mayinclude a laser diode, a vertical cavity surface emitting laser, a lightemitting diode, a tunable laser, or some other light source that emitscoherent or partially coherent light. The projector assembly 210 emitslight in a visible band (e.g., from about 390 nm to 700 nm), and it mayemit light that is continuous or pulsed. In some embodiments, theprojector assembly 210 may be a laser that emits light at a particularwavelength (e.g., 532 nanometers). The projector assembly 210 emitslight in accordance with one or more illumination parameters receivedfrom the display controller 250. An illumination parameter is aninstruction used by the projector assembly 210 to generate light. Anillumination parameter may include, e.g., source wavelength, pulse rate,pulse amplitude, beam type (continuous or pulsed), other parameter(s)that affect the emitted light, or some combination thereof.

The waveguide assembly 220 outputs image light into an eye of a user.The image light may be generated by the projector assembly 210. In someembodiments, the generated image light may be incoupled (e.g., via oneor more gratings) into the waveguide assembly 220. The waveguideassembly 220 may then output the incoupled image light into the eyebox.The waveguide assembly 220 includes one or more waveguides. The one ormore waveguides are configured to receive image light from the projectorassembly 210 and project the image light for a pupil replication in aneyebox. In some embodiments, the display assembly 200 includes aseparate waveguide for each color channel. Alternatively, the waveguideassembly 220 may incouple image light from each color channel into asingle waveguide. The waveguide assembly 220 may outcouple the incoupledlight to the eyebox via one or more gratings. The gratings of thewaveguide assembly 220 may include a diffraction grating, a holographicgrating, a holographic reflector, or a combination thereof.

The waveguide assembly 220 may be composed of one or more materials thatfacilitate total internal reflection of the generated image light. Forexample, a grating of the waveguide assembly 220 that couples thegenerated image light into a waveguide may be a diffraction gratinghaving a pitch in the range of 300 nanometers (nm) to 600 nm for totalinternal reflection. Similarly, a grating of the waveguide assembly 220that decouples image light out of the waveguide can be a diffractiongrating with a pitch configured to cause incident image light to exitthe waveguide (e.g., the pitch having a range of 300 to 600 nm). Awaveguide of the waveguide assembly 220 may be composed of e.g.,silicon, plastic, glass, or polymers, or some combination thereof. Thewaveguide assembly 220 can have a relatively small form factor. Forexample, the waveguide assembly 220 may be approximately 50 mm widealong x-dimension, 30 mm long along y-dimension and 0.2-1 mm thick alongz-dimension. The display assembly 200 may include multiple waveguideassemblies. For example, a first waveguide assembly may output imagelight into a left eye of a user while a second waveguide assembly mayoutput image light into a right eye of the user.

The piezoelectric movement sensors 230 may measure and monitor movementor displacement of one or more waveguides of the waveguide assembly 220.The composition of a piezoelectric movement sensor may include apiezoelectric thin-film that is layered between a top electrode and abottom electrode. In some embodiments, an electrode of the piezoelectricmovement sensor has a length and/or width that ranges from onemillimeter to five centimeters. The terms “top” and “bottom” are usedfor convenience and should not require orientation of the piezoelectricmovement sensors 230. The electrodes may be composed of a metal nanowireink, indium tin oxide (ITO), poly(3,4-ethylenedioxythiophene)polystyrene sulfonate (PEDOT:PSS), any suitable material for transparentelectrodes, or a combination thereof. The piezoelectric thin-film may becomposed of a polymer such as polyvinylidene fluoride-trifluoroethylene(PVDF-TrFE), any suitable flexible piezoelectric material, or acombination thereof. The piezoelectric thin-film may cover some or allof a surface of the waveguide assembly 220. In some embodiments, eachpiezoelectric movement sensor of the sensors 230 may have a separate setof electrodes. By contrast, additionally or alternatively, thepiezoelectric movement sensors 230 may share one or more electrodes. Forexample, a bottom electrode may be shared between two or more of thesensors 230, where the two or more sensors have separate top electrodeswith the shared bottom electrode. An example composition of apiezoelectric movement sensor is described in the description of FIG. 3. Further, the movement sensors are referred to throughout aspiezoelectric movement sensors; however, the movement sensors formonitoring the movement of a waveguide assembly may include or may bereplaced with piezoresistive movement sensors. The change in resistivityin response to a force caused by the movement of the waveguide assembly220 may be used to monitor and correct for the movement. Additionalexamples of sensors include resistive sensors, inductive sensors,capacitive sensors, any suitable force or displacement sensor formonitoring waveguide movement (e.g., having sub-micron precision), or acombination thereof.

The piezoelectric movement sensors 230 may be co-located with thewaveguide assembly 220 or at a location of the display assembly 200proximal to the waveguide assembly 220. For example, the piezoelectricmovement sensors 230 may be co-located at the periphery of a surface ofthe waveguide assembly 220. In another example, the piezoelectricmovement sensors 230 may be located at a frame around the waveguide orat an area connecting waveguides (e.g., nose bridge of artificialreality glasses). The piezoelectric movement sensors 230 may be locatedat one or more surfaces of the waveguide assembly 220. Surfaces caninclude a surface proximal to an environment and a surface proximal tothe user (i.e., with reference to when the display assembly is in use).The piezoelectric movement sensors 230 may be configured to have anorientation for measuring movement along that orientation. In oneexample, a piezoelectric movement sensor includes a top electrode thatis shaped in a line or rectangle in a particular direction (e.g.,horizontally in a line parallel to a line connecting the user's eyes).This piezoelectric movement sensor is configured to monitor for movementoccurring in the same direction (e.g., a movement horizontally along thesurface of the waveguide on which the piezoelectric movement sensorresides). The piezoelectric movement sensors 230 may have the sameorientation (e.g., all oriented vertically). In some embodiments, thepiezoelectric movement sensors 230 may be oriented orthogonal to oneanother (e.g., in vertical and horizontal directions). The piezoelectricmovement sensors 230 may be oriented radially with respect to a centerof the waveguide assembly 220 (e.g., radially from a portion of thewaveguide that overlaps the user's pupil when the user wears a headsetincluding the display assembly 200). Example locations andconfigurations of the piezoelectric movement sensors are described inthe description of FIG. 4 .

The piezoelectric movement sensors 240 may be substantially transparent.As referred to herein, a value described using “approximately” or“substantially” may be construed as having a range within +/−10% of thevalue unless another meaning is apparent from the context. For example,“approximately ten” should be understood to mean “in a range from nineto eleven.” In another example, substantially transparent can refer to atransmission of light through the piezoelectric movement sensors that iswithin 90-100% light transmission. In some embodiments, thepiezoelectric movement sensors 240 within a potential field of view of auser are substantially transparent to visible light. For example, asubset of the piezoelectric movement sensors 240 adjacent to a border ofan eyebox of the display assembly 200 are within a potential field ofview and are substantially transparent. In some embodiments, at leastone of the piezoelectric movement sensors 240 is outside of a field ofview of the user and is opaque or substantially opaque to visible light.Substantially opaque may refer to a transmission of light through thepiezoelectric movement sensors that is within 10% of 0% lighttransmission. For example, as depicted in FIG. 1B, a portion of thedisplay assembly 120 along the outer border of display elements (e.g.,lenses) may be substantially opaque.

The temperature sensor 240 measures a temperature associated with themovement of the waveguide assembly 220. A temperature that the waveguideassembly is operating under may negatively affect the waveguide assembly220 (e.g., heat expands or bends the shape of a waveguide). Examples oftemperatures associated with the waveguide assembly include atemperature of a component of the display assembly 200, a temperature ofan environment in which the display assembly 200 operates, any suitabletemperature measurement associated with the operation of the displayassembly 200, or combination thereof. Examples of temperature sensorsinclude thermocouples, thermistors, resistance temperature detector(RTD), infrared sensors, any suitable sensor for measuring temperature,or a combination thereof. A temperature sensor 240 may have a formfactor, location, or combination thereof suitable for measuring acomponent of the display assembly 200. The display assembly 200 mayinclude multiple temperature sensors in addition to the temperaturesensor 240. For example, a first temperature sensor may be locatedproximal to the projector assembly 210 and a second temperature sensormay be located proximal to one of the piezoelectric movement sensors230. Similarly, the display assembly 200 may include both one or moreco-located temperature sensors and one or more remotely locatedtemperature sensors. In some embodiments, temperature sensed by thetemperature sensor 240 may be used to determine an accuracy with whichthe piezoelectric movement sensor 240 monitors for movements in thewaveguide assembly 220. For example, the sensitivity of a material thatmay be included in the piezoelectric movement sensor 240 (e.g., a PVDFpolymer) can experience a degradation in sensitivity commensurate withan increase in temperature. The temperature sensor 240 may provide ameasured temperature to the display controller 250 to determine that thetemperature has exceeded a threshold temperature for accuracy of thepiezoelectric movement sensors 230 and in response, determine not tocorrect an amount of movement because the amount determined may beinaccurate.

In a first example of a configuration of the temperature sensor 240, thetemperature sensor 240 is a surface thermocouple may be coupled to thewaveguide assembly 220 to measure the temperature at a portion of awaveguide of the display assembly 200. In a second example of aconfiguration of the temperature sensor 240, the temperature sensor 240is a thermistor is embedded within a nose bridge connecting twowaveguide assemblies of the display assembly 200. The temperaturemeasured at this location connecting two waveguide assemblies may beassociated with a relative movement between the two waveguideassemblies. Although depicted as co-located with the display assembly200, the temperature sensor 240 may alternatively be located remote fromthe display assembly 200. The remotely located temperature sensor may becommunicatively coupled to the display assembly 200 via communicationscircuitry at the display assembly 200 or a device to which the displayassembly 200 is coupled. For example, temperature measured by a nationalweather service is provided to a user's mobile phone (e.g., at a weatherapplication), communicated to a headset of the user via Bluetoothcircuitry at the headset, and accessed by the display assembly 200(e.g., by the display controller) to determine a correction foraberration in image light based on movement monitored by the movementsensors and the accessed temperature. The use of a temperature sensor tocorrect for an amount of aberration is further described in thedescription of FIG. 5 .

The display controller 250 can generate instructions to cause one ormore actuators to mechanically deform or move the waveguides of thewaveguide assembly 220. Examples of actuators includemicroelectromechanical systems (MEMS) actuators (e.g., electrostatic,electrothermal, electromagnetic, and piezoelectric actuation). Theactuators may be located proximal to the piezoelectric movement sensors230. Alternatively or additionally, the actuators may be located aroundthe waveguide assembly 220 (e.g., in the frame of a headset in which thedisplay assembly 200 is included). The actuators may be substantiallytransparent to increase light transmission efficiency.

The display controller 250 controls the display operations of thedisplay assembly 200. The controller 250 determines display instructionsfor the waveguide assembly 220. Display instructions are instructions torender one or more images. In some embodiments, display instructions maysimply be an image file (e.g., bitmap). The display instructions may bereceived from, e.g., a console of a VR system. The controller 250includes a combination of hardware, software, and/or firmware not shownhere so as not to obscure other aspects of the disclosure. The displaycontroller 250 can determine instructions for correcting an amount ofaberration in projected image light caused by movement in a waveguideassembly. These instructions are referred to as “movement instructions.”The display controller 250 may determine movement instructions using atleast a monitored movement of the waveguide assembly. For example, thedisplay controller 250 may determine a mapping of a movement instructionto a monitored movement or an estimated amount of aberration in theimage light caused in part by the monitored movement.

The display controller 250 may determine an estimated amount ofaberration based on at least the movement monitored by the piezoelectricmovement sensors 230. The display controller 250 may receive sensormeasurements from sensors in addition to the piezoelectric movementsensors 220. Such supplemental sensors may include temperature sensors(e.g., the temperature sensor 240), motion sensors, location sensors,proximity sensors, any suitable sensor measuring data affecting theperformance of the waveguide assembly or the measurement of the movementof the waveguide assembly, or a combination thereof waveguide assembly220 movement In an example of using a supplemental sensor, thetemperature sensor 240 is a supplemental sensor. The display controller250 may receive a temperature measurement from the temperature sensor240 and access a set of models representative of various waveguidemovements measured under the received temperature measurement. Varioussets of models for movements measured under respective temperatures maybe stored at a storage of the display assembly 200 or a device in whichthe display assembly 200 is included (e.g., a headset) or stored at aremote server communicatively coupled to the display assembly.

In some embodiments, the display controller 250 may use a supplementalsensor to determine when to check for a movement in the waveguideassembly 220. In some embodiments, the display controller 250 mayperiodically check for movements while the display assembly 200 is inactive use (e.g., while the user is using an artificial realityapplication). Additionally or alternatively, the display controller 250may check for movement in response to receiving a measurement from asupplemental sensor indicating that a movement has likely occurred. Thedisplay controller 250 may receive the measurement, and in response,determine a likelihood that the movement has occurred based on thereceived measurement. For example, in response to receiving ameasurement from a motion sensor that the display assembly 200 has beensubject to a rapid acceleration and sudden stop (e.g., corresponding toa drop of the headset), the display controller 250 may use a model(e.g., statistical model or machine learning model) to determine alikelihood that an event that can cause a movement in the waveguide hasoccurred. In response to determining that the likelihood meets orexceeds a threshold likelihood, the display controller 250 may proceedto use the piezoelectric movement sensors 240 to measure movement in thewaveguide assembly 220. In response to determining that the likelihooddoes not exceed the threshold likelihood, the display controller 250 maydetermine not to check for a movement at that time. By reducing thenumber of instances during which movement in the waveguide assembly ismonitored (e.g., using a predetermined frequency or upon the occurrenceof a condition such as detecting a drop), the display assembly 200 mayreduce the power resources consumed relative to continuously monitoringfor movement in the waveguide assembly. Reducing the power consumed maybe especially beneficial for mobile or wireless devices integrating thedisplay assembly 200 (e.g., all-day wearable AR glasses) that rely on abattery.

The display controller 250 may determine an estimated accuracy level ofthe monitored movement based on supplemental sensors. In someembodiments, the display controller 250 may use the temperature measuredby the temperature sensor 240 to determine the estimated accuracy level.The display controller 250 may receive a temperature measurement andcompare the temperature measurement to a threshold temperature ortemperature range. The threshold temperature may represent a minimumtemperature (e.g., 150 degrees Celsius or approximately 300 degreesFahrenheit) at which the measurements by the piezoelectric movementsensors cannot be used. The temperature range may represent a range oftemperatures at which the measurements can be used (e.g., between −20and 149 degrees Celsius). In response to the comparison, the displaycontroller 250 may determine to proceed to with determining a correctionto an amount of aberration in projected image light or not to proceedwith determining the correction due to a likely inaccurate movementmeasurement caused by temperature distortion.

In some embodiments, the display controller 250 may use eye trackinginformation to monitor for movement in the waveguide assembly. Forexample, the eye tracking information, representative of the user's gazeat a projected image distorted due to an aberration in the waveguideassembly, may be used in combination with information from thepiezoelectric movement sensors 240 to determine, by the displaycontroller 250, a likelihood that a movement has occurred at aparticular location of the waveguide assembly.

The display controller 250 may correct for aberrations in image lightaffected by movements of the waveguide assembly 220 using one or more ofa software correction or mechanical correction. The correction may berepresented through one or more instructions executable by components ofthe display assembly 200. In some embodiments, the display controller250 inputs the detected movement of the waveguide assembly 220 monitoredby the piezoelectric movement sensors 230 into a model (e.g., astatistical model or a machine-learned model). The model can estimate anamount of aberration (e.g., blurring, distortion, chromatic aberration,etc.) caused by the movement of the waveguide assembly 220. The displaycontroller 250 uses the estimated amount of aberration to determine amovement instruction for correcting the detected movement. Softwaremovement instructions may include modifications to the image lightgenerated by the projector assembly 210. These modifications canmitigate the estimated amount of aberration. Mechanical movementinstructions may include modifications to a shape of the waveguideassembly 220. Mechanical movement instructions may cause actuators tomove one or more waveguides of the waveguide assembly 220 back into ashape or position that allows for the display of an accurate image tothe user. The display controller 250 may input the detected movement ofthe waveguide assembly 220 into a model, which may be the same or adifferent model as referenced previously with respect to determining anestimated amount of aberration, and receive as output from the model, anestimated amount of misalignment of the waveguide assembly 220. Thedisplay controller 250 may then use the estimated amount of misalignmentto determine mechanical movement instructions for actuators to move thewaveguide assembly 220 into a position that allows for the user to viewcorrectly projected images. Although not depicted in FIG. 2 , thedisplay assembly 200 may include actuators that modify the shape of awaveguide of the waveguide assembly 220 according to the movementinstructions.

In some embodiments, the display controller 250 may compare adisplacement or movement as measured by the piezoelectric movementsensors 230 to a predetermined movement model. One example of apredetermined model may include features from a Zernike model ofaberrations to estimate an amount of aberration. The display controller250 may access a Zernike model of aberrations associated with distortionin the sagittal direction (i.e., the “vertical tilt” or “Y-tilt” model),determine a similarity between the measured movement and a Zernike modelof an aberration, and in response to determining that a thresholdsimilarity has been met, apply predetermined movement instructionsassociated with the vertical tilt distortion. The movement instructionsmay be one or more of modifying the image light generated by theprojector assembly 210 or modifying the shape of the waveguide assembly220 to correct for the aberration in the vertical direction.

The display controller 250 may determine a movement instruction thatcorrects for binocular disparity (e.g., misalignment) between twowaveguides (e.g., waveguides for the left and right eyes). The displaycontroller 250 may receive measurements from one or more piezoelectricmovement sensors monitoring a relative movement between a firstwaveguide assembly and a second waveguide assembly of the displayassembly 200. The display controller 250 can determine movementinstructions modifying at least one of a first image light projected bythe first waveguide assembly or a second image light projected by thesecond waveguide assembly based on the relative movement. The projectorassembly 210 may then execute the movement instructions to generate theat least one of the modified first or second image lights. The at leastone of the modified first image light or the modified second image lightcan correct for aberration affecting the display of a three dimensionalartificial reality object that the first and second image lights weregenerated to display.

FIG. 3 is a top view 300 a and cross-sectional view 300 b of a displayassembly 300, in accordance with one or more embodiments. Thecross-sectional view 300 b is taken from a line 335 shown in the topview 300 a. The top view 300 a of the display assembly 300 showspiezoelectric movement sensors layered over a waveguide assembly 330 fora left eye. The waveguide assembly 330 is coupled to a projectorassembly 302.

The projector assembly 302 projects image light into the waveguideassembly 330 for displaying an image to the user. Light from theprojector assembly 302 may be incoupled into the waveguide assembly 330(via one or more gratings) and outcoupled (via a different set of one ormore gratings) toward an eyebox. The projector assembly 302 may be anembodiment of the projector assembly 210. The projector assembly 302 mayinclude light sources in three different color channels (e.g., red,green, and blue) that generate image light.

The waveguide assembly 330 outputs image light into an eye of the user.Image light generated by the projector assembly 302 may be received bythe waveguide assembly through one or more gratings and output towardsan eyebox through a different set of gratings. The waveguide assembly330 may include multiple waveguides. For example, the waveguide assembly330 may include a separate waveguide for each color channel.

Piezoelectric movement sensors of the display assembly 300 monitor formovement of the waveguide assembly 330 and provide monitored movement toa display controller for correcting an amount of aberration in imagelight caused by the movement. The piezoelectric movement sensors mayeach include a top electrode 305, a piezoelectric thin-film 315, and abottom electrode 325. The display assembly 300 shows a center region ofthe waveguide assembly that is circumscribed by a peripheral region. Inthe illustrated embodiment, a piezoelectric thin-film 315 of thepiezoelectric movement sensors is located in the peripheral region andnot the center region. While the piezoelectric thin-film 315 is depictedas a continuous layer in the peripheral region, in alternativeembodiments, the piezoelectric thin-film 315 may be separated intodiscrete areas layered over a bottom electrode and the waveguideassembly 330. For example, substantially under each of the topelectrodes only. This configuration may involve an increase infabrication complexity, but may reduce the impact of the piezoelectricmovement sensors on an image quality. A layer of the piezoelectricthin-film 315 may decrease light transmission efficiency through thearea where the layer exists. The separation of the thin-film 315 intodiscrete areas enables an improvement in light transmission efficiencyrelative to a layer of thin-film over an entirety of the waveguideassembly 330.

The piezoelectric movement sensors are further depicted as extendingover the nose bridge of the headset in which the display assembly 300 isincluded. The piezoelectric movement sensors extending over the nosebridge and in contact with both the waveguide assemblies for the leftand right eyes may measure a relative movement between the two waveguideassemblies (e.g., a misalignment). The top-view 300 a shows topelectrodes of the piezoelectric movement sensors in a circular orientedin the peripheral region. The number, spacing, size, and orientation ofthe top electrodes may be different than what is depicted in top view300 a. For example, there may be a horizontally oriented top electrodethat spans the top border of the peripheral region (i.e., proximal tothe top of the user's head when the headset is worn) in place of the twotop electrodes depicted at the top border in top view 300 a.

The cross-section 300 b of the display assembly 300 shows piezoelectricmovement sensors coupled to a waveguide 329 of the display assembly 300via one or more intermediate layers. The piezoelectric movement sensorsare located at the side of the display assembly 300 closer to theenvironment (i.e., distal from an eye of a user). The side closer to theenvironment may be referred to as the “world-side” or as “an outwardfacing area.” The side closer to the user may be referred to as the“eye-side” or as “an inward facing area.” Each piezoelectric movementsensor may include a bottom electrode 325, piezoelectric thin film 315,and a top electrode 305. In some embodiments, the piezoelectric movementsensors may be at one or more of the world-side or eye-side. The topelectrode 305 and the bottom electrode 325 may be composed of one ormore of a metal nanowire ink, indium tin oxide (ITO), orpoly(3,4-ethylenedioxythiophene) polystyrene sulfonate (PEDOT:PSS).Optionally, a polyethylene terephthalate (PET) substrate may be includedadjacent to the bottom electrode 325 (e.g., below the bottom electrode325, proximate to the user's eye). The waveguide assembly 330 includesthe waveguide 329. A waveguide may be composed of a silicon carbide(SiC) substrate. While a single waveguide grating is shown, in otherembodiments there may be waveguide and grating for each color channel(e.g., vertically stacked).

The intermediate layers may include an output grating 328, an overcoat327, and an anti-reflective coating (ARC) 326. This configuration oflayers may be present at the center region of the waveguide assembly330. In some embodiments, the intermediate layers may be different,fewer, or greater. For example, at the peripheral regions of thewaveguide assembly 330, the intermediate layers may include only theovercoat 327 and the ARC 326, but not the output grating 328. As shownin the cross-section 300 b, the output grating 328 is not layeredbeneath the piezoelectric movement sensors that are located at theperipheral regions of the waveguide assembly 330. In some embodiments,the piezoelectric movement sensors may couple directly to a surface ofthe waveguide absent the intermediate layers. One or more of the topelectrode 305, the piezoelectric thin film 315, or the bottom electrode325 may be substantially transparent to light in the visible band.

FIG. 4 depicts movement sensor configurations, in accordance withvarious embodiments. In a configuration 400, piezoelectric movementsensors are located only over a peripheral region of a waveguideassembly 425. The piezoelectric thin-film layer 415 is layered over theentirety of the peripheral region while top electrodes 405 are inseparated locations throughout the peripheral region. The top electrodes405 are shown in vertical, horizontal, and diagonal orientations. Inparticular, the vertically oriented electrodes 445, horizontallyoriented electrodes 435, and diagonally oriented electrodes 455 areshown in the configurations in FIG. 4 . The vertically oriented topelectrodes 445 may monitor vertical movements. The horizontally orientedtop electrodes 435 may monitor horizontal movements. The diagonallyoriented top electrodes 455 may monitor both horizontal and verticalmovements. The size of the top electrodes 405 are depicted assubstantially consistent across each top electrode. In alternativeembodiments, the size of the top electrodes 405 may be different fromone another.

The dimensions of the electrodes may be sized such that movement of awaveguide assembly can be measured while also differentiating the shapeof deformations. An electrode having a dimension that is sized too large(e.g., an electrode that is greater than 5 centimeters in length and/orwidth while not extending beyond the dimensions of the waveguideassembly on which it is overlaid) may be unable to provide a measurementof deformation that distinguishes the deformation's shape. An electrodehaving a dimension that is sized too small (e.g., a non-zero length thatis less than one millimeter) may be unable to detect that a deformationhas occurred. Although the electrodes in FIG. 4 are depicted as beingrectangular in shape, the electrodes may have different shapes (e.g.,circular, elliptical, triangular, or any suitable polygonal shape).

In a configuration 410, piezoelectric movement sensors are similarlylocated and oriented to the configuration 400. However, thepiezoelectric thin-film layer 415 is located over all of the waveguideassembly 425 rather than only over the peripheral region. Theconfiguration 410 may have a benefit over the configuration 400 withrelation to the ease of manufacturing. For example, manufacturing alayer of piezoelectric thin-film over the entirety of the waveguideassembly 425 may be less complicated than manufacturing a layer that isonly at the peripheral region.

In a configuration 420, the piezoelectric thin-film layer 415 is alsolocated over all of the waveguide assembly 425. The configuration of thepiezoelectric movement sensors has changed from the sensorconfigurations in the configurations 400 and 410. The configuration 420includes a greater number of piezoelectric movement sensors than in theconfigurations 400 and 410. Additionally, a length of the top electrodesis not consistent across each electrode. For example, electrodes 450 arelonger than the electrode 451. Certain top electrodes (e.g., electrodes450) are evenly distributed radially around a center 465 of thewaveguide assembly 425 and may have larger dimensions than other topelectrodes (e.g., the top electrodes 405) shown in the configurations400 and 410. An electrode whose dimensions cover more surface area cancapture more information regarding movement of the waveguide assembly.This benefit may be considered alongside a possibility that too large ofan electrode may be unable to measure a distinguishable shape in awaveguide assembly's deformation. A subset of the top electrodes 405proximal to the horizontal edges of the waveguide assembly 425 areoriented horizontally and a subset of the top electrodes 405 proximal tothe vertical edges of the waveguide assembly 425 are orientedvertically. Similar to the configurations 400 and 410, the configuration420 includes the top electrodes 405 located around the center of thewaveguide assembly 425 such that the center does not include topelectrodes 405. The omission of top electrodes 405 from the center ofthe waveguide assembly 425 may improve the light transmission efficiencythrough the center of the waveguide assembly 425.

In a configuration 430, the piezoelectric movement sensors are locatedexclusively on the nose bridge connecting two waveguide assemblies forthe left and right eyes. While the piezoelectric movement sensors in theconfiguration 430 are shown in a horizontal orientation, the orientationmay additionally or alternatively be vertical, diagonal, or acombination of horizontal, vertical, and diagonal orientations.

Although FIG. 4 shows the configurations 400, 410, 420, and 430separately, one or more of the configurations may be combined. Forexample, the configuration 400 may be combined with the configuration430 such that the piezoelectric movement sensors are located at theedges of the waveguides for both eyes in addition to the nose bridgeconnecting the two waveguides.

FIG. 5 is a block diagram 500 of a feedback process for correctingaberrations monitored by piezoelectric movement sensors, in accordancewith one or more embodiments. A display controller 502 is coupled to atemperature sensor 501, movement sensors 505 a and 505 b, and projectorassemblies 503 a and 503 b. The projector assembly 503 a is used togenerate image light for display to a right eye of a user and is coupledto a waveguide 504 a that outputs the image light to an eyebox 506 a.The projector assembly 503 b is used to generate image light for displayto a left eye of the user and is coupled to a waveguide 504 b thatoutputs the image light to an eyebox 506 b. The movement sensor 505 amonitors for movements in the waveguide 504 a and the movement sensor505 b monitors for movements in the waveguide 504 b.

The display controller 502 provides instructions to the projectorassemblies 503 a and 503 b, which generate image light that is projectedthrough the waveguides 504 a and 504 b, respectively. The movementsensors 505 a and 505 b feed back monitored movements in the waveguides504 a and 504 b, respectively, to the display controller 502. Thedisplay controller 502 corrects for an amount of aberration in one ormore of the waveguide 504 a or 504 b based on at least the fed backmovement measurements. The display controller 502 may use one or moremodels to estimate an amount of aberration in image light or an amountof misalignment of the waveguides 504 a or 504 b using the fed backmovement measurements. Based on the estimates output from the model(s),the display controller 502 determines instructions for software and/ormechanical correction to the waveguides 504 a or 504 b. As illustrated,the display controller 502 can provide instructions to the projectorassemblies 503 a and 503 b to generate modified image light such thataberration caused by the movement is offset. Furthermore, although notdepicted, in some embodiments, there may also be actuators thatphysically deform and/or position one or more of the projectorassemblies 503 a and 503 b and/or one or more of the waveguides 504 aand 504 b to offset the aberration.

In some embodiments, the display controller 502 uses additional sensorinformation to correct for the amount of aberration. For example, thetemperature of the display assembly in which the waveguides 504 a and504 b reside or the environment surrounding the display assembly ismeasured by a temperature sensor 501 and provided to the displaycontroller 502. The display controller 502 can use the measuredtemperature to determine an accuracy level of the measured movement, asthe accuracy of movement sensors, such as piezoelectric sensors,decreases as temperature increases. Additional or alternative sensorssupplementing the movement sensors 505 a and 505 b can include motionsensors, location sensors (e.g., Global Positioning System sensors), anysuitable sensor measuring data impacting the performance of the movementsensors or the movement of the waveguides, or a combination thereof.

While the feedback process of FIG. 5 shows the components of a displayassembly used to improve the display of images at a user's eye, thefeedback process may similarly be used to provide corrected image lightto a scanning system. In an example of the display assembly in ascanning system, the display assembly may include a light sourcegenerating beams of light (e.g., lasers) to scan an object (e.g., athree dimensional object). The projector assemblies 503 a and 504 b mayinclude, for example, lasers. The beams of light may travel through thewaveguides 504 a and 504 b, and the movement sensors 505 a and 505 b mayprovide a measurement of movement in the waveguides 504 a and 504 b tothe display controller 502 determine an amount of aberration in thebeams of light that negatively impacts the scanning of the object. Inone example of an effect of waveguide movement in a scanning system,movement in the waveguides can change the expected time of flightthrough the light beams through waveguides (towards the scanned object,reflected from the scanned object, or both), where the expected time offlight through the waveguides is used to determine a depth of the 3Dobject's surface. The display controller 502 may use the measuredmovement to determine movement instructions for the amount of aberrationin the beams of light produced by the lasers or reflected from thescanned object. Thus, in this scanning system example, the eyeboxes 506a and 506 b may be replaced with the object to be scanned or replacedwith one or more camera sensors for receiving beams of light reflectedfrom the scanned object.

FIG. 6 is a flowchart illustrating a process for correcting an amount ofaberration of a waveguide assembly, in accordance with one or moreembodiments. The process shown in FIG. 6 may be performed by componentsof a display assembly (e.g., the display assembly 200). Other entitiesmay perform some or all of the steps in FIG. 6 in other embodiments.Embodiments may include different and/or additional steps, or performthe steps in different orders. For example, the display assembly maymeasure movement in a waveguide assembly prior to the projection ofimage light through the waveguide assembly.

The display assembly projects 610 image light from a waveguide assemblytowards an eyebox. The display controller may instruct the projectorassembly to generate image light. For example, while the user of aheadset having the display assembly is interacting with an artificialreality application, the display controller instructs the projectorassembly to render a three dimensional image of an object at the eyeboxfor display to the user.

The display assembly monitors 620 the movement of the waveguideassembly. Piezoelectric movement sensors may measure a change in theshape of the waveguide assembly (e.g., caused by an electric shock,thermal stress, or a drop of the display assembly). Additionally oralternatively, the display assembly may monitor for displacement of thewaveguide assembly (e.g., movement due to a loose or otherwisedeteriorating frame of the headset configured to hold the waveguideassembly in place). The piezoelectric movement sensors may be configuredaround a peripheral region of a waveguide (e.g., the configuration 400of FIG. 4 ). Additional piezoelectric movement sensors may be located ata nose bridge of the headset to monitor for relative movement betweentwo waveguide assemblies of the headset (e.g., respective waveguideassemblies for outputting image light to the left and right eyes tocreate a three dimensional image).

The display assembly corrects 630 for aberration in the image lightbased in part on the monitored movement. A display controller of thedisplay assembly may receive the measured movement monitored by thepiezoelectric movement sensors. The display controller may determinemovement instructions to correct for an amount of aberration in theimage light projected 610 using at least the monitored movement. Thedisplay controller may use one or more models to estimate an amount ofaberration in image light or an amount of misalignment of the waveguideassembly using the movement measured by the piezoelectric movementsensors. Based on the estimates output from the model(s), the displaycontroller can determine instructions for software and/or mechanicalcorrection (e.g., via actuators) to the waveguide assembly. In someembodiments, the display controller may additionally use a temperatureat one or more waveguide assemblies, as measured by a temperaturesensor, to determine an estimated accuracy of the monitored movement, anestimated amount of aberration, or combination thereof.

FIG. 7 is a system 700 that includes a headset 705, in accordance withone or more embodiments. In some embodiments, the headset 705 may be theheadset 100 of FIG. 1A or FIG. 1B. The system 700 may operate in anartificial reality environment (e.g., a virtual reality environment, anaugmented reality environment, a mixed reality environment, or somecombination thereof). The system 700 shown by FIG. 7 includes theheadset 705, an input/output (I/O) interface 710 that is coupled to aconsole 715 and the network 720. While FIG. 7 shows an example system700 including one headset 705 and one I/O interface 710, in otherembodiments any number of these components may be included in the system700. For example, there may be multiple headsets each having anassociated I/O interface 710, with each headset and I/O interface 710communicating with the console 715. In alternative configurations,different and/or additional components may be included in the system700. Additionally, functionality described in conjunction with one ormore of the components shown in FIG. 7 may be distributed among thecomponents in a different manner than described in conjunction with FIG.7 in some embodiments. For example, some or all of the functionality ofthe console 715 may be provided by the headset 705.

The headset 705 includes the display assembly 730, one or more positionsensors 740, and the DCA 745. The display assembly 730 may include allor a subset of the components of the display assembly 200. The displayassembly 730 can include a waveguide assembly, one or more piezoelectricmovement sensors coupled to the waveguide assembly, and a displaycontroller. The waveguide assembly of the display assembly 730 isconfigured to project image light towards an eyebox. Movement of thewaveguide assembly may contribute at least in part to an amount ofaberration in the projected image light. The display assembly 730includes one or more piezoelectric movement sensors configured tomonitor the movement of the waveguide assembly. The display controlleris configured to correct for the amount of aberration in the image lightbased in part on the monitored movement. Some embodiments of headset 705have different components than those described in conjunction with FIG.7 . Additionally, the functionality provided by various componentsdescribed in conjunction with FIG. 7 may be differently distributedamong the components of the headset 705 in other embodiments, or becaptured in separate assemblies remote from the headset 705.

The display assembly 730 displays content to the user in accordance withdata received from the console 715. The display assembly 730 displaysthe content using one or more display elements (e.g., the displayelements 120). A display element may be, e.g., an electronic display. Invarious embodiments, the display assembly 730 comprises a single displayelement or multiple display elements (e.g., a display for each eye of auser). Examples of an electronic display include: a liquid crystaldisplay (LCD), an organic light emitting diode (OLED) display, anactive-matrix organic light-emitting diode display (AMOLED), a waveguidedisplay, some other display, or some combination thereof.

The display assembly 730 may magnify image light received from theelectronic display, corrects optical errors associated with the imagelight, and presents the corrected image light to one or both eyeboxes ofthe headset 705. In various embodiments, the display assembly 730includes one or more optical elements. Example optical elements includedin the display assembly 730 include: an aperture, a Fresnel lens, aconvex lens, a concave lens, a filter, a reflecting surface, or anyother suitable optical element that affects image light. Moreover, thedisplay assembly 730 may include combinations of different opticalelements. In some embodiments, one or more of the optical elements inthe display assembly 730 may have one or more coatings, such aspartially reflective or anti-reflective coatings.

Magnification and focusing of the image light by the display assembly730 allows the electronic display to be physically smaller, weigh less,and consume less power than larger displays. Additionally, magnificationmay increase the field of view of the content presented by theelectronic display. For example, the field of view of the displayedcontent is such that the displayed content is presented using almost all(e.g., approximately 110 degrees diagonal), and in some cases, all ofthe user's field of view. Additionally, in some embodiments, the amountof magnification may be adjusted by adding or removing optical elements.

In some embodiments, the display assembly 730 may be designed to correctone or more types of optical error. Examples of optical error includebarrel or pincushion distortion, longitudinal chromatic aberrations, ortransverse chromatic aberrations. Other types of optical errors mayfurther include spherical aberrations, chromatic aberrations, or errorsdue to the lens field curvature, astigmatisms, or any other type ofoptical error. In some embodiments, content provided to the electronicdisplay for display is pre-distorted, and the display assembly 730corrects the distortion when it receives image light from the electronicdisplay generated based on the content.

The position sensor 740 is an electronic device that generates dataindicating a position of the headset 705. The position sensor 740generates one or more measurement signals in response to motion of theheadset 705. The position sensor 190 is an embodiment of the positionsensor 740. Examples of a position sensor 740 include: one or more IMUS,one or more accelerometers, one or more gyroscopes, one or moremagnetometers, another suitable type of sensor that detects motion, orsome combination thereof. The position sensor 740 may include multipleaccelerometers to measure translational motion (forward/back, up/down,left/right) and multiple gyroscopes to measure rotational motion (e.g.,pitch, yaw, roll). In some embodiments, an IMU rapidly samples themeasurement signals and calculates the estimated position of the headset705 from the sampled data. For example, the IMU integrates themeasurement signals received from the accelerometers over time toestimate a velocity vector and integrates the velocity vector over timeto determine an estimated position of a reference point on the headset705. The reference point is a point that may be used to describe theposition of the headset 705. While the reference point may generally bedefined as a point in space, however, in practice the reference point isdefined as a point within the headset 705.

The DCA 745 generates depth information for a portion of the local area.The DCA includes one or more imaging devices and a DCA controller. TheDCA 745 may also include an illuminator. Operation and structure of theDCA 745 is described above with regard to FIG. 1A.

The I/O interface 710 is a device that allows a user to send actionrequests and receive responses from the console 715. An action requestis a request to perform a particular action. For example, an actionrequest may be an instruction to start or end capture of image or videodata, or an instruction to perform a particular action within anapplication. The I/O interface 710 may include one or more inputdevices. Example input devices include: a keyboard, a mouse, a gamecontroller, or any other suitable device for receiving action requestsand communicating the action requests to the console 715. An actionrequest received by the I/O interface 710 is communicated to the console715, which performs an action corresponding to the action request. Insome embodiments, the I/O interface 710 includes an IMU that capturescalibration data indicating an estimated position of the I/O interface710 relative to an initial position of the I/O interface 710. In someembodiments, the I/O interface 710 may provide haptic feedback to theuser in accordance with instructions received from the console 715. Forexample, haptic feedback is provided when an action request is received,or the console 715 communicates instructions to the I/O interface 710causing the I/O interface 710 to generate haptic feedback when theconsole 715 performs an action.

The console 715 provides content to the headset 705 for processing inaccordance with information received from one or more of: the DCA 745,the headset 705, and the I/O interface 710. In the example shown in FIG.7 , the console 715 includes an application store 755, a tracking module760, and an engine 765. Some embodiments of the console 715 havedifferent modules or components than those described in conjunction withFIG. 7 . Similarly, the functions further described below may bedistributed among components of the console 715 in a different mannerthan described in conjunction with FIG. 7 . In some embodiments, thefunctionality discussed herein with respect to the console 715 may beimplemented in the headset 705, or a remote system.

The application store 755 stores one or more applications for executionby the console 715. An application is a group of instructions, that whenexecuted by a processor, generates content for presentation to the user.Content generated by an application may be in response to inputsreceived from the user via movement of the headset 705 or the I/Ointerface 710. Examples of applications include: gaming applications,conferencing applications, video playback applications, or othersuitable applications.

The tracking module 760 tracks movements of the headset 705 or of theI/O interface 710 using information from the DCA 745, the one or moreposition sensors 740, or some combination thereof. For example, thetracking module 760 determines a position of a reference point of theheadset 705 in a mapping of a local area based on information from theheadset 705. The tracking module 760 may also determine positions of anobject or virtual object. Additionally, in some embodiments, thetracking module 760 may use portions of data indicating a position ofthe headset 705 from the position sensor 740 as well as representationsof the local area from the DCA 745 to predict a future location of theheadset 705. The tracking module 760 provides the estimated or predictedfuture position of the headset 705 or the I/O interface 710 to theengine 765.

The engine 765 executes applications and receives position information,acceleration information, velocity information, predicted futurepositions, or some combination thereof, of the headset 705 from thetracking module 760. Based on the received information, the engine 765determines content to provide to the headset 705 for presentation to theuser. For example, if the received information indicates that the userhas looked to the left, the engine 765 generates content for the headset705 that mirrors the user's movement in a virtual local area or in alocal area augmenting the local area with additional content.Additionally, the engine 765 performs an action within an applicationexecuting on the console 715 in response to an action request receivedfrom the I/O interface 710 and provides feedback to the user that theaction was performed. The provided feedback may be visual or audiblefeedback via the headset 705 or haptic feedback via the I/O interface710.

The network 720 couples the headset 705 and/or the console 715 to themapping server 725. The network 720 may include any combination of localarea and/or wide area networks using both wireless and/or wiredcommunication systems. For example, the network 720 may include theInternet, as well as mobile telephone networks. In one embodiment, thenetwork 720 uses standard communications technologies and/or protocols.Hence, the network 720 may include links using technologies such asEthernet, 802.11, worldwide interoperability for microwave access(WiMAX), 2G/3G/4G mobile communications protocols, digital subscriberline (DSL), asynchronous transfer mode (ATM), InfiniBand, PCI ExpressAdvanced Switching, etc. Similarly, the networking protocols used on thenetwork 720 can include multiprotocol label switching (MPLS), thetransmission control protocol/Internet protocol (TCP/IP), the UserDatagram Protocol (UDP), the hypertext transport protocol (HTTP), thesimple mail transfer protocol (SMTP), the file transfer protocol (FTP),etc. The data exchanged over the network 720 can be represented usingtechnologies and/or formats including image data in binary form (e.g.Portable Network Graphics (PNG)), hypertext markup language (HTML),extensible markup language (XML), etc. In addition, all or some of linkscan be encrypted using conventional encryption technologies such assecure sockets layer (SSL), transport layer security (TLS), virtualprivate networks (VPNs), Internet Protocol security (IPsec), etc.

One or more components of system 700 may contain a privacy module thatstores one or more privacy settings for user data elements. The userdata elements describe the user or the headset 705. For example, theuser data elements may describe a physical characteristic of the user,an action performed by the user, a location of the user of the headset705, a location of the headset 705, an HRTF for the user, etc. Privacysettings (or “access settings”) for a user data element may be stored inany suitable manner, such as, for example, in association with the userdata element, in an index on an authorization server, in anothersuitable manner, or any suitable combination thereof.

A privacy setting for a user data element specifies how the user dataelement (or particular information associated with the user dataelement) can be accessed, stored, or otherwise used (e.g., viewed,shared, modified, copied, executed, surfaced, or identified). In someembodiments, the privacy settings for a user data element may specify a“blocked list” of entities that may not access certain informationassociated with the user data element. The privacy settings associatedwith the user data element may specify any suitable granularity ofpermitted access or denial of access. For example, some entities mayhave permission to see that a specific user data element exists, someentities may have permission to view the content of the specific userdata element, and some entities may have permission to modify thespecific user data element. The privacy settings may allow the user toallow other entities to access or store user data elements for a finiteperiod of time.

The privacy settings may allow a user to specify one or more geographiclocations from which user data elements can be accessed. Access ordenial of access to the user data elements may depend on the geographiclocation of an entity who is attempting to access the user dataelements. For example, the user may allow access to a user data elementand specify that the user data element is accessible to an entity onlywhile the user is in a particular location. If the user leaves theparticular location, the user data element may no longer be accessibleto the entity. As another example, the user may specify that a user dataelement is accessible only to entities within a threshold distance fromthe user, such as another user of a headset within the same local areaas the user. If the user subsequently changes location, the entity withaccess to the user data element may lose access, while a new group ofentities may gain access as they come within the threshold distance ofthe user.

The system 700 may include one or more authorization/privacy servers forenforcing privacy settings. A request from an entity for a particularuser data element may identify the entity associated with the requestand the user data element may be sent only to the entity if theauthorization server determines that the entity is authorized to accessthe user data element based on the privacy settings associated with theuser data element. If the requesting entity is not authorized to accessthe user data element, the authorization server may prevent therequested user data element from being retrieved or may prevent therequested user data element from being sent to the entity. Although thisdisclosure describes enforcing privacy settings in a particular manner,this disclosure contemplates enforcing privacy settings in any suitablemanner.

Additional Configuration Information

The foregoing description of the embodiments has been presented forillustration; it is not intended to be exhaustive or to limit the patentrights to the precise forms disclosed. Persons skilled in the relevantart can appreciate that many modifications and variations are possibleconsidering the above disclosure.

Some portions of this description describe the embodiments in terms ofalgorithms and symbolic representations of operations on information.These algorithmic descriptions and representations are commonly used bythose skilled in the data processing arts to convey the substance oftheir work effectively to others skilled in the art. These operations,while described functionally, computationally, or logically, areunderstood to be implemented by computer programs or equivalentelectrical circuits, microcode, or the like. Furthermore, it has alsoproven convenient at times, to refer to these arrangements of operationsas modules, without loss of generality. The described operations andtheir associated modules may be embodied in software, firmware,hardware, or any combinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allthe steps, operations, or processes described.

Embodiments may also relate to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, and/or it may comprise a general-purpose computingdevice selectively activated or reconfigured by a computer programstored in the computer. Such a computer program may be stored in anon-transitory, tangible computer readable storage medium, or any typeof media suitable for storing electronic instructions, which may becoupled to a computer system bus. Furthermore, any computing systemsreferred to in the specification may include a single processor or maybe architectures employing multiple processor designs for increasedcomputing capability.

Embodiments may also relate to a product that is produced by a computingprocess described herein. Such a product may comprise informationresulting from a computing process, where the information is stored on anon-transitory, tangible computer readable storage medium and mayinclude any embodiment of a computer program product or other datacombination described herein.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the patent rights. It istherefore intended that the scope of the patent rights be limited not bythis detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thepatent rights, which is set forth in the following claims.

What is claimed is:
 1. A display assembly comprising: a waveguideassembly configured to project image light towards an eyebox, wherein anamount of aberration in the image light is based in part on a movementof the waveguide assembly; one or more piezoelectric movement sensorscoupled to the waveguide assembly, the one or more piezoelectricmovement sensors configured to monitor the movement of the waveguideassembly; and a display controller configured to correct for the amountof aberration in the image light based in part on the monitoredmovement.
 2. The display assembly of claim 1, wherein the one or morepiezoelectric movement sensors comprises: a plurality of electrodes; anda piezoelectric thin film.
 3. The display assembly of claim 2, whereinthe piezoelectric thin film is a continuous layer of a piezoelectricmaterial.
 4. The display assembly of claim 2, wherein the piezoelectricthin film is a plurality of discrete layers of a piezoelectric material.5. The display assembly of claim 1, wherein each electrode of aplurality of electrodes has a length that ranges from 2-10 millimeters.6. The display assembly of claim 1, wherein the one or morepiezoelectric movement sensors are oriented in a plurality oforientations, wherein a first orientation of the plurality oforientations is orthogonal to a second orientation of the plurality oforientations.
 7. The display assembly of claim 1, wherein the one ormore piezoelectric movement sensors are oriented in a plurality oforientations, wherein an orientation of the plurality of orientations isradial with respect to a center of the waveguide assembly.
 8. Thedisplay assembly of claim 1, wherein the waveguide assembly includes acenter region that is circumscribed by a peripheral region, and apiezoelectric thin film of the one or more piezoelectric movementsensors is located in the peripheral region and not the center region.9. The display assembly of claim 1, wherein a piezoelectric thin film ofthe one or more piezoelectric movement sensors is layered over anoutward facing area of the waveguide assembly proximal to anenvironment, wherein an inward facing area of the waveguide assembly isproximal to a user.
 10. The display assembly of claim 1, wherein thedisplay assembly is coupled to a headset.
 11. The display assembly ofclaim 1, wherein the one or more piezoelectric movement sensors is afirst set of piezoelectric movement sensors and wherein the waveguideassembly is a first waveguide assembly for a first eye of a user, thedisplay assembly further comprising: a second set of piezoelectricmovement sensors coupled to a bridge connecting the first waveguideassembly to a second waveguide assembly for a second eye of the user,the second set of piezoelectric movement sensors configured to monitor arelative movement between the first waveguide assembly and the secondwaveguide assembly.
 12. The display assembly of claim 11, wherein thedisplay controller is further configured to: modify at least one of afirst image light projected by the first waveguide assembly or a secondimage light projected by the second waveguide assembly based on therelative movement, wherein at least one of the modified first imagelight or the modified second image light corrects for aberrationaffecting display of a three-dimensional artificial reality object usingthe first image light and the second image light.
 13. The displayassembly of claim 1, further comprising: a projector assembly comprisinga plurality of light sources that generate the image light, wherein thewaveguide assembly includes one or more waveguides configured to receivethe image light and project the image light for a pupil replication inthe eyebox; and wherein the display controller is further configured tocause the projector assembly to correct the image light to mitigate theamount of aberration.
 14. The display assembly of claim 1, wherein thewaveguide assembly includes one or more waveguides, and the displayassembly further comprises: actuators configured to modify a shape of awaveguide of the one or more waveguides; and wherein the displaycontroller is further configured to: determine movement instructions formodifying the shape of the waveguide based on the monitored movement ofthe waveguide assembly; and instruct the actuators to modify the shapeof the waveguide in accordance with the movement instructions.
 15. Thedisplay assembly of claim 1, further comprising: a temperature sensorconfigured to determine a temperature of an environment in which thedisplay assembly operates; and wherein the display controller is furtherconfigured to correct for the amount of aberration in the image lightbased on the temperature.
 16. A method comprising: projecting imagelight from a waveguide assembly towards an eyebox, wherein an amount ofaberration in the image light is based in part on a movement of thewaveguide assembly; monitoring, via one or more piezoelectric movementsensors coupled to the waveguide assembly, the movement of the waveguideassembly; and correcting for the amount of aberration in the image lightbased in part on the monitored movement.
 17. The method of claim 16,wherein the one or more piezoelectric movement sensors is a first set ofpiezo electric movement sensors and wherein the waveguide assembly is afirst waveguide assembly for a first eye of a user, the method furthercomprising: monitoring a relative movement between the first waveguideassembly and a second waveguide assembly for a second eye of the user,wherein a second set of piezoelectric movement sensors are coupled to abridge connecting the first waveguide assembly to the second waveguideassembly; and monitoring at least one of a first image light projectedby the first waveguide assembly or a second image light projected by thesecond waveguide assembly based on the relative movement, wherein atleast one of a modified first image light or a modified second imagelight corrects for aberration affecting display of a three-dimensionalartificial reality object using the first image light and the secondimage light.
 18. The method of claim 16, further comprising: generatingthe image light using a plurality of light sources of a projectorassembly; receiving, at one or more waveguides of the waveguideassembly, the image light; projecting, at the one or more waveguides,the image light for a pupil replication in the eyebox; and causing theprojector assembly to correct the image light to mitigate the amount ofaberration.
 19. The method of claim 16, wherein the waveguide assemblyincludes one or more waveguides, and the method further comprising:modifying a shape of a waveguide of the one or more waveguides;determining movement instructions for modifying the shape of thewaveguide based on the monitored movement of the waveguide assembly; andinstructing actuators to modify the shape of the waveguide in accordancewith the movement instructions.
 20. A non-transitory computer-readablestorage medium comprising stored instructions, the instructions whenexecuted by a processor of a device, causing the device to: projectimage light from a waveguide assembly towards an eyebox, wherein anamount of aberration in the image light is based in part on a movementof the waveguide assembly; monitor, via one or more piezoelectricmovement sensors coupled to the waveguide assembly, the movement of thewaveguide assembly; and correct for the amount of aberration in theimage light based in part on the monitored movement.